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Objectives 

This  research  was  an  initial  exploration  into  the  ability  of  a  team  of  autonomous  software  agents  to  be 
effective  in  unknown  and  changing  optimization  environments  by  evolving  to  use  the  most  successful 
algorithms  at  the  points  in  the  optimization  process  where  they  will  be  the  most  effective.  The  goal  of 
this  project  was  to  define  the  core  framework  and  methodology  for  agent-based  adaptive  optimization 
strategies  using  an  evolutionary  approach  at  the  strategic,  rather  than  solution  level  -  where  the 
strategies  of  agents  in  the  team  (the  decisions  for  picking,  altering,  and  inserting  a  solution)  evolve  over 
time.  As  one  application  of  this  approach,  individual  solutions  arc  tours  in  the  familiar  combinatorial 
optimization  problem  of  the  traveling  salesman.  As  another  application,  an  initial  3  dimensional  layout 
problem  is  defined.  The  work  also  sets  the  stage  to  begin  investigating  how  teams  coordinate. 


Accomplishments/New  Findings 

There  are  several  factors  motivating  a  strategy  evolution  approach  to  solving  complex 
optimization  problems  in  dynamic  environments.  Firstly,  because  changing  variables  means  that  the 
structure  and  dimension  of  the  design  space  changes  over  time,  a  flexible  representation  of  solutions  is 
needed.  Because  solution  structure  is  not  static,  it  follows  that  the  strategies  for  generating  solutions 
should  not  be  fixed  either,  but  should  instead  adapt  to  the  design  space  as  it  changes.  Also,  since  the 
topology  (the  constraints  and  objectives)  of  the  problem  design  space  are  constantly  changing,  it  is  also 
impractical  to  rely  on  a  single  algorithm  or  strategy,  especially  since  the  more  complex  the  algorithm, 
the  more  fine-tuning  is  needed  to  make  it  effective  on  a  specific  instance  of  the  problem. 

In  the  evolutionary  multi-agent  systems  (EMAS)  framework,  we  achieve  adaptive  strategy 
application  by  allowing  solution  strategics  to  evolve  over  the  course  of  the  optimization.  Each  strategy 
for  generating  new  solutions  is  embodied  in  a  single,  autonomous  agent.  The  agents  (strategics)  work  in 
parallel  to  produce  new  solutions  and  explore  the  design  space,  thus  eliminating  reliance  on  a  single 
algorithm.  In  addition,  agents  cooperate  by  sharing  the  results  of  their  work  in  databases  or  ‘memories’, 
to  which  all  other  agents  have  access  to.  The  shared  memories  may  contain  partial  and  working 
solutions  in  addition  to  complete  solutions  and  while  the  quality  of  solutions  generated  drives  the 
evolutionary  process  of  the  agents,  the  agents  are  not  endowed  with  a  priori  knowledge  of  the  solution 
structure.  Thus,  changing  the  structure  (the  number  of  variables)  of  the  solution  docs  not  adversely 
affect  the  agents’  ability  to  create  new  solutions. 

As  agents  create  new  solutions  and  improve  old  ones,  they  are  rewarded  or  punished  for  their 
successes  and  failures.  The  reward  or  punishment  is  translated  into  a  change  in  the  agent’s  fitness. 
Over  the  course  of  the  optimization,  agents  may  reproduce  if  they  have  been  successful  (i.e.  if  they  have 
a  high  fitness),  or  may  be  removed  from  the  population  if  they  have  not.  Thus,  genes  in  the  agents’ 
chromosomes  representing  pieces  of  an  entire  strategy  (either  learned  or  inherited)  are  passed  from  one 
generation  to  the  next  through  genetic  operators  like  crossover  and  mutation.  The  result  is  a  constantly 
changing  team  of  heterogeneous  strategies  that  produces  -  as  a  byproduct  of  its  actions  -  a  continuously 
improving  set  of  solutions  in  the  shared  memory. 

The  results  present  a  convincing  argument  for  the  evolution  of  agents  in  a  team  at  the  population 
level.  Decisions  have  proven  to  be  a  useful  genetic  property  of  agents  in  such  an  evolutionary  setting. 
The  evolutionary  teams  evolved  to  generate  better  solutions  than  the  base  algorithms  alone.  We  have 
also  shown  that  the  strength  of  the  EMAS  algorithm  lies  in  its  ability  to  evolve  the  best  team  of  agents 
dynamically.  Evolution  and  activation  within  this  team  results  in  solutions  that  are  better  than  simply 
running  the  same  number  of  algorithms  randomly  on  a  similar  set  of  solutions.  We  thus  argue  that  the 
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use  of  evolutionary  agents  to  determine  the  best  solution  strategies  dynamically  is  a  strong  approach  to 
adaptive  optimization.  Decentralizing  the  strategy  selection  process  and  allowing  strategics,  rather  than 
solutions,  to  progress  in  an  evolutionary  manner  means  that  a  much  broader  range  of  design  and 
optimization  applications,  including  layout,  scheduling,  and  manufacturing  planning,  can  be  confronted 
in  the  face  of  design  space  uncertainty  and  change.  Another  strength  of  the  EMAS  algorithm  is  as  a 
predictive  or  learning  guide  for  which  set  of  algorithms  or  strategics  should  or  should  not  be  employed 
and  when.  Utilizing  EMAS  in  this  way,  at  least  as  seen  on  a  static  case  of  the  TSP,  has  been  shown  to 
lower  computation  time  while  maintaining  or  even  improving  solution  quality. 

As  a  next  step  we  have  created  a  foundation  to  explore  EMAS  on  a  complex  engineering  design 
problem,  the  3  dimensional  layout  of  a  product.  We  have  implemented  4  optimization  algorithms: 
simulated  annealing,  extended  pattern  search,  genetic  algorithm,  and  the  Powcl’s  Method  local  search. 
EMAS  determines  which  solution  strategics  the  agents  will  use  at  any  time,  and  parameters  for  those 
algorithms.  We  have  tested  the  approach  on  a  simple  cube  packing  problem  and  the  layout  of  an  engine 
compartment  for  a  vehicle.  Initial  results  are  quite  promising  and  will  be  thoroughly  analyzed  and 
compared  to  both  the  TSP  findings,  and  the  results  of  individual  algorithms. 

We  have  also  begun  to  use  EMAS  to  consider  how  teams  effectively  communicate,  what  strategies 
work  best  and  how  that  influences  EMAS  results 
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